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1.  INTRODUCTION 


Adaptive  filtering  is  a  very  powerful  technique  that  is  currently  used  for  many  military  and 
commercial  applications.  This  technique  allows  one  to  change  the  control  or  interface  as  a  function 
of  the  surrounding  environment.  The  basic  idea  is  to  extract  enough  information  from  particular 
signals  to  create  a  plant/model  that  emulates  the  resultant  output  signal  as  close  as  possible.  One  of 
most  common  uses  of  this  type  of  digital  signal  processing  (DSP)  is  for  signal  identification.  Figure 
1  shows  a  very  basic  example  of  this  type  of  system. 


Figure  1.  Basic  system  identification  using  adaptive  filters  block  diagram. 

The  objective  is  to  adjust  the  adaptive  finite-impulse  response  (AFIR)  filter’s  coefficients  until  the 
error  signal  e(n)  is  minimized  to  some  predetermined  level.  This  error  signal  is  simply  the 
magnitude  difference  sum  of  the  unknown  signal  to  be  identified,  y(n),  and  its  corresponding 
emulated  signal  y  ’(n).  Statistical  algorithms  such  as  the  least  mean  square  (LMS)  algorithm,  or 
deterministic  approaches,  such  as  the  recursive  least-squares  (RLS),  are  usually  employed  inside  the 
AFIR  block  shown  in  Figure  1  to  adjust  the  finite-impulse  response  (FIR)  coefficients. 

Most  adaptive  filtering  applications  concentrate  on  optimizing  the  coefficients  of  FIR  filters. 

Various  iterative  techniques  (e.g.,  LMS  using  gradient  descent)  are  employed  to  search  for  the 
optimal  solution.  In  the  current  investigation,  a  new  adaptive  filtering  method  is  introduced  that 
uses  GAs  to  modify  the  coefficient  sets  representing  a  selected  wavelet  to  minimize  the  quantization 
error  of  the  reconstructed  signal.  Unlike  the  signal  identification  process,  however,  this  process 
requires  access  to  all  signal  information.  Consequently,  this  approach  establishes  a  new  method  for 
using  GAs  to  evolve  sets  of  wavelet  coefficients  that  outperform  standard  wavelet  transforms  used 
for  reconstruction  of  one-dimensional  (1-D)  and  two-dimensional  (2-D)  data  subjected  to 
quantization  noise. 

This  research  purposely  chooses  to  concentrate  all  efforts  on  the  lossy  compression  of  signals,  due 
to  the  fact  that  original  signal  information  is  permanently  lost  during  the  quantization  process.  This 
approach  consistently  identifies  coefficient  sets  that  significantly  reduce  MSE  and  improve  PSNR 
for  lossy  compression  systems  based  on  the  wavelet  transform.  A  simple  block  diagram  of  our 
system  is  depicted  in  Figure  2. 
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e(k) 


Figure  2.  Basic  block  diagram  of  adaptive  wavelet  filtering  system. 

The  main  objective  of  the  system  shown  in  Figure  2  is  to  minimize  the  error  signal,  e(k),  which  is  a 
function  of  the  magnitude  differences  between  the  original  uncorrupted  signal  v(k)  and  the 
transformed,  downsampled,  quantized,  dequantized,  upsampled,  and  finally  inverse  transformed 
v(k)’  signal.  In  this  effort,  the  results  of  the  error  signal  e(k)  is  used  by  a  GA  to  adjust  the  filter 
coefficients  of  a  preselected  wavelet.  The  results  of  the  GA  process  are  then  used  to  generate  v(k)\ 
This  entire  process  is  contained  within  what  is  called  the  “Adaptive  Discrete  Wavelet  Transform” 
(ADWT)  plant  shown  in  Figure  2. 

Following  the  introduction,  a  background  about  previous  DWT  research  is  presented  along  with 
basic  DWT  and  Quantization  noise  theory.  The  main  research  objective  is  then  stated  followed  by  a 
short  demonstration  of  how  sensitive  the  DWT  reconstruction  process  is  to  changes  in  its  filter 
coefficients.  Then  a  brief  introduction  into  basic  GA  theory  is  presented.  Finally,  the  experimental 
results  are  presented  followed  by  conclusions  and  proposed  future  research. 

Note  that,  throughout  this  report,  any  reference  to  the  wavelet  forward  and  inverse  transforms  is 
termed  as  "standard"  transforms  such  as  the  Harr  and  Daub4  wavelet  transforms.  Whereas  the  term 
“evolved”  pertains  only  to  the  altered  wavelet  coefficients  used  during  the  inverse  transform 
process. 


2.  BACKGROUND 

The  image  processing  research  conducted  at  the  AFRL/IFTA  Reconfigurable  Computing 
Laboratory  has  been  successful  in  several  areas.  For  example,  several  CAD  programs  have  been 
developed  to  provide  a  means  of  efficiently  evaluating  various  wavelet-based  compression 
algorithms.  This  software  permits  the  user  to  manually  select  standard  wavelets  for  a  particular 
application  (e.g.,  SAR  image  compression).  The  hope  is  that  the  results  are  ultimately  deemed 
acceptable  for  utilization  in  various  unmanned  aerial  vehicle  imaging  systems.  Once  airborne,  the 
transforms  are  to  be  used  on-board  the  airframe  to  compress  and  transmit  imagery  to  the  remote 
ground  station  for  viewing  and  evaluation. 
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The  applications  developed  in  this  research  effort  use  wavelet  filters  that  have  been  previously 
discovered  by  other  works  such  as  that  accomplished  by,  for  example,  Harr  in  1920  and  Daubechies 
in  1992.  The  wavelet  selection  criterion  is  traditionally  a  subjective  decision  based  upon  both  the 
type  of  data  to  be  processed  and  the  type  of  processing.  With  a  limited  number  of  wavelet  filters,  it 
is  unknown  whether  other  filters  may  perform  much  better,  depending  upon  the  particular 
environment  through  which  the  system  is  operating  and  the  mission  to  be  accomplished. 
Furthermore,  a  previously  selected  wavelet  may  not  work  as  well  in  a  particular  environment  where 
other  wavelets  may  be  better  suited.  Thus,  a  chosen  wavelet  may  only  work  satisfactorily  for 
specific  periods  of  time,  and  the  wavelet  may  not  be  optimized  when  scenes  change  (e.g.,  from 
ground  to  air  or  ground  to  water).  This  problem  is  the  main  impetus  behind  the  current  research. 


3.  THE  DISCRETE  WAVELET  TRANSFORM 

Fourier  analysis  (Stein  and  Weiss  1971)  is  based  upon  the  idea  that  all  continuous  signals  may  be 
represented  by  a  sum  of  infinitely  repeating  sines  and  cosines.  Fourier  transforms  and  their  variants 
[e.g.,  the  Fast  Fourier  Transform  (Cooley  and  Tukey  1965)]  provide  information  about  the 
frequencies  present  in  a  signal,  but  fail  to  describe  where  and  when  those  frequencies  occur  in  either 
time  or  space.  In  contrast,  wavelets  transform  time-domain  signals  to  a  joint  time-frequency 
domain.  A  wavelet  transform  convolves  a  given  signal  with  particular  instances  of  the  wavelet  at 
various  time  scales  and  positions.  Since  changes  in  frequency  may  be  modeled  by  changing  the  time 
scale,  and  time  changes  may  be  modeled  by  shifting  the  position  of  the  wavelet,  wavelets  allow  us 
to  represent  both  the  frequencies  within  a  given  signal  and  the  location  of  those  frequencies. 
However,  unlike  the  sinusoidal  functions  involved  in  the  Fourier  transform,  wavelets  exist  only 
within  finite  intervals,  and  are  zero-valued  outside  of  those  intervals. 

The  DWT  is  used  to  redistribute  the  energy  of  a  given  signal.  The  two  main  components  of  the 
classical  DWT  are  (1)  the  scaling  function  and  (2)  the  wavelet  function  y/(t) ,  which  are  defined 
as  follows: 


=  X  hJ(2t~n) 

n 


(i) 


r(0  =  Z  (2) 

n 

where 

hn  =  impulse  response  of  the  scaling  filter, 

gn  =  impulse  response  of  the  wavelet  filter, 

n  =  denotes  the  scale  and  shift  of  these  functions. 

Moreover,  hn  contains  the  set  of  filter  coefficients  corresponding  to  the  projection  of  the  basis 
functions  for  the  low-pass  filtering  section  of  the  DWT,  and  gn  contains  filter  coefficients 
corresponding  to  the  projection  of  the  basis  functions  for  the  high-pass  filtering  section  of  the  DWT. 
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Once  transformed,  the  analysis  of  signal  input  x(t)  results  in  discrete  sets  of  data  in  the  wavelet 
domain. 

The  inverse  DWT  (DWT1)  is  used  to  transform  coefficients  from  the  wavelet  domain  back  into  the 
original  signal  domain.  In  other  words,  the  inverse  transform  produces  the  original  signal  x(t)  from 
the  wavelet  and  scaling  coefficients. 


+00  +00 

40  =£  2X„rM(0  (3) 

£=-oo  n=- oo 

where 

+oo 

dk,n  =  \  V  *k,n  (Ox(t)dt  (4) 

—  00 


A  DWT  perfectly  describes  a  continuous-time  signal  x(t)  using  a  countable  set  of  coefficients.  The 
DWT  shown  in  Figure  3  is  a  multi-pass  process  that  uses  a  scaling  function  and  a  wavelet  function 
to  separate  the  high-pass  and  low-pass  information  content  of  a  given  signal. 

The  low-pass  sub-band  produced  by  the  previous  transform  is  then  subdivided  into  its  own  low  and 
high  sub-bands  by  the  next  level  of  the  transform.  Using  the  multiresolution  analysis  (MRA) 
technique  (Mallat  1989),  this  process  could  be  repeated  as  many  times  as  the  signal  length  can  be 
divided  by  two  (Walker  1999,  p.  16);  in  practice,  however,  it  is  common  to  perform  only  a 
relatively  small  number  of  iterations  (i.e.,  levels  of  decomposition),  since  nearly  all  of  the  energy  of 
the  decomposition  coefficients  is  concentrated  in  the  lower  subbands  (Rajoub  2002). 


B  B 


Figure  3.  (Forward)  DWT  with  Multi-resolution 
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Standard  wavelets  may  be  described  by  sets  of  coefficients  (Daubechies  1992,  p.  54-56),  commonly 
named  g  and  h  for  the  high-pass  and  low-pass  filters,  respectively.  For  example,  the  Haar  and 
Daubechies-4  (Daub4)  wavelets  are  defined  in  Table  1. 


Table  1.  Mathematical  Definitions  for  Haar  and  Daub4  Wavelets 


Haar 

Daub4 

*"{; Tr-T?} 

g(n)  =  -rh[P-n] 

h- 

g(n) 

V2*(l  +  V3)  V2*(3  +  V3)  a/2*(3-V3)  V2*(l-V3)' 
[  8  ’  8  ’  8  ’  8  J 

i  =  -rh[P-n] 

■ 

where  n  is  the  (spatial)  position,  and  P  is  the  filter  length.  (P  =  2  for  the  Haar  wavelet  and  P  =  4  for 
the  Daub4  wavelet.)  Although  Haar  wavelets  (Haar  1910)  are  the  simplest  wavelets  to  describe, 
Daubechies  wavelets  (Daubechies  1992)  are  more  commonly  used  in  certain  applications.  Figure  4 
is  a  graphic  illustration  of  the  time-domain  behavior  of  each  of  the  wavelets  defined  in  Table  1. 
Daub4  wavelets  use  filter  banks  (Vaidyanathan  1992)  containing  exactly  four  elements. 


Haar 


wavelet 


scaling 


Figure  4.  Time-Domain  Representation  of  Wavelet  and  Scaling  Function  of  Haar  and  Daub4 

Wavelets 


An  N-stage  1-D  DWT  is  depicted  in  Figure  5. 
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Figure  5.  N-Stage,  1-D  Analysis  DWT  Filter 

A  forward  wavelet  transform  can  be  performed  via  matrix  multiplication.  As  an  illustrative 
example,  consider  a  1-D  scenario  that  uses  a  Daub4  filter  and  an  input  signal  x  of  length  eight.  If 
the  four  vectors  (a  and  d)  comprising  the  low-pass  and  high-pass  sections  are  combined  to  make  a 
single  eight-row  vector,  then  the  following  matrix  product  defines  the  forward  DWT: 

A3  hi  hi  hO  0  0  0  OTx(O) 

0  0  A3  A2  A1  AO  0  0  x(l) 

0  0  0  0  A3  hi  hi  AO  x(2) 

hi  AO  0  0  0  0  A3  A2  x(3) 

<?3  gl  gl  gO  0  0  0  0  x(4) 

0  0  gl  gl  gl  gO  0  0  x(5) 

0  0  0  0  gl  gl  g\  gO  x(6) 

gl  gO  0  0  0  0  g3  gl_  _x(7) 

The  equivalent  C  language  algorithm  that  illustrates  a  single  step  of  a  1-D  forward  DWT  utilizing  a 
Daub4  wavelet  is  as  follows: 

N  =  8; 

for  (i  =0;  i  <  (N/2);  i++) 

{ 

a  =  ( 2 i  +  2)  %  N; 
b  =  (2i  +  3)  %  N; 

a [ i ]  =  x[2i]*h[3]  +  x[2i+l]*h[2]  +  x[a]*h[l]  +  x[b]*h[0]; 
d [ i ]  =  x [ 2i ] *g [ 3 ]  +  x[2i+l]*g[2]  +  x[a]*g[l]  +  x[b]*g[0]; 

} 
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where  x  is  the  input  signal,  N  is  the  length  of  that  signal,  g  is  the  wavelet  filter,  and  h  is  the  scaling 
filter.  The  resulting  high-pass  ( d)  and  low-pass  (a)  signals  each  contain  N/2  values.  Additional 
steps  repeat  this  process  using  only  the  low-pass  data;  for  each  additional  step,  n  will  be  half  its 
value  from  the  previous  step. 

An  n-stage  1-D  inverse  DWT'1  is  depicted  in  Figure  6. 


Figure  6.  N-Stage,  1-D  Analysis  Synthesis  DWT'1  Filter. 

Similarly,  the  1-D  inverse  DWT'1  for  our  previous  matrix  example  can  be  expressed  as  follows: 


x(0) 

Zr3 

0 

0 

hi 

g3 

0 

0 

gl 

a(0) 

x(l) 

hi 

0 

0 

hO 

g2 

0 

0 

go 

a(l) 

x(2) 

hi 

Zr3 

0 

0 

gl 

g3 

0 

0 

a(2) 

x(3) 

hO 

hi 

0 

0 

go 

8  2 

0 

0 

a  (3) 

x(4) 

0 

hi 

Zr3 

0 

0 

gl 

g3 

0 

d(  0) 

x(5) 

0 

hO 

hi 

0 

0 

go 

g2 

0 

d(  1) 

x(6) 

0 

0 

hi 

hi 

0 

0 

gl 

g3 

d(  2) 

x(7)_ 

_  0 

0 

hO 

hi 

0 

0 

g0 

g2_ 

_d(  3) 

The  following  C  language  algorithm  describes  the  1-D  inverse  DWT'1  for  a  Daub4  wavelet  used  in 
this  matrix  example: 

N  =  8; 

for  (i  =0;  i  <  (N/2);  i++) 

{ 

x [ 2i  ]  =  a [ i-1 ] *h [ 1 ]  +  a[i]*h[3]  +  d[N-l]*g[l]  +  d[N]*g[3]; 
x  [  2i  +  l ]  =  a [N-l ] *h [ 0 ]  +  a[N]*h[2]  +  d[N-l]*g[0]  +  d[N]*g[2]; 
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where  x  is  the  reconstructed  output  signal,  N  is  the  discrete  length  of  that  signal,  g  is  the  wavelet 
filter,  and  h  is  the  scaling  filter.  The  resulting  high-pass  ( d)  and  low-pass  (a)  signals  each  contain 
N/2  values.  Additional  steps  repeat  this  process  using  only  the  low-pass  data.  For  each  additional 
step,  N  will  be  half  its  value  from  the  previous  step.  It  is  important  to  note  that  a  separable  2-D 
transform  on,  for  example,  visible  images  is  accomplished  by  performing  a  1-D  transform  on  each 
row,  followed  by  a  1-D  transform  on  each  column. 

Lossy  Wavelet-based  compression  methods  are  capable  of  much  greater  compression  gain  than  can 
be  achieved  by  lossless  methods.  However,  a  fundamental  theorem  from  information  theory 
(Shannon  and  Weaver  1964)  places  limits  on  the  amount  of  compression  that  can  be  achieved  by  a 
lossless  encoding  technique.  This  additional  compression  comes  at  the  price  of  introducing 
inaccuracies  into  the  reconstructed  signal  (Walker  1999,  p.  19).  As  such,  wavelets  (Graps  1995) 
provide  a  powerful  technique  for  efficiently  representing  large  amounts  of  data. 

Note  that  “perfect  reconstruction”  of  1-D  signals  or  2-D  images  is  only  possible  due  to  the 
properties  of  the  orthonormal  basis  vectors  used  by  the  DWT  process.  Examples  of  orthogonal 
functions  are  trigonometric  and  exponential  functions  (trigonometric  functions  are  currently  being 
used  for  DCT-based  compression  methods  such  as  JPEG). 

Two  or  more  vectors  are  orthonormal  if  and  only  if  their  total  dot  product  is  zero,  i.e.,  each  vector 
must  have  a  zero  component  in  the  direction  of  every  other  vector.  Assume  that  we  are  working  in 
2-D  rectangular  space  and  we  define  that  space  with  two  unit  basis  vectors: 

ax,ay  (5) 

The  two  unit  vectors  form  an  orthonormal  set  if,  for  all  x,  y: 

ax  •  ay  =  0  (6) 


4.  QUANTIZATION  NOISE 

For  DSP  applications,  quantization  of  data  is  an  absolute  necessity  through  analog-to-digital 
conversion.  In  these  situations,  quantization  noise  occurs  when  the  number  of  bits  used  is 
insufficient  for  representing  the  full  dynamic  range  of  the  signal.  In  lossy  compression  methods, 
quantization  is  used  to  decrease  the  dynamic  range  of  the  signal  reducing  the  amount  of  data 
necessary  to  represent  the  signal.  For  example,  a  32-bit  signed  integer  (whose  value  may  range 
from  -2  to  +2  -  1)  may  be  quantized  to  16-bit  values  (ranging  from  -2  to  +2  -  1);  in  this  case, 
quantization  reduces  the  amount  of  information  of  the  resulting  signal  up  to  a  factor  of  65536.  In 
effect,  the  information  contained  in  the  least  significant  16  bits  of  the  original  signal  is  lost.  Figure 
7  shows  a  1-D  example  of  this  process. 
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Figure  7.  Quantization  example. 

As  can  be  seen,  the  staircase  function  resulting  from  quantization  is  not  equivalent  to  the  original 
continuous  function.  The  difference  between  the  original  function’s  magnitude  and  the  quantized 
one  at  each  discrete  point  is  defined  as  the  quantization  error. 

Quantization  usually  involves  one  of  two  processes:  either  (1)  truncation  or  (2)  rounding.  To 
truncate,  one  simply  discards  bits  to  the  right  of  the  least  significant  bit.  To  round,  one  looks  at  the 
value  of  the  most  significant  bit  of  that  portion  of  the  signal  which  is  being  eliminated;  the  value  of 
the  remaining  signal  is  incremented  if  and  only  if  this  bit  is  set.  Quantization  is  not  an  additive 
process  and  in  fact  discards  (sometimes  valuable)  information.  In  addition,  quantization  is 
inherently  a  non-linear  process  which  makes  accurate  reconstruction  of  the  original  signal  more 
difficult. 

Haar,  Daubechies,  and  other  families  of  wavelets  have  certain  desirable  properties  that  make  them 
very  useful  for  image  compression.  First,  these  wavelets  conserve  energy:  that  is,  the  total  amount 
of  energy  from  the  original  signal  is  retained  in  the  wavelet  domain  (Walker  1999,  pp.  6-7,  9,  38- 
40).  Second,  these  wavelets  redistribute  the  energy  from  the  original  signal,  such  that  nearly  all  of 
that  energy  is  stored  in  the  first  trend  subsignal,  a1;  most  of  the  remaining  values  of  the  wavelet 
coefficients  will  be  insignificant,  and  may  be  eliminated  without  significant  loss  of  information 
(Saha  2000). 

In  a  typical  wavelet  application,  a  signal  is  forward  wavelet  transformed  by  a  wavelet  and  is  then 
quantized  (Q)  immediately  prior  to  an  encoding  step  (E)  for  compression.  The  results  are  then 
transmitted  over  a  wire,  air,  etc,  medium.  On  the  receiving  end,  the  incoming  signal  is  decoded  (D), 
dequantized  (Q1)  and  then  an  inverse  wavelet  transform  is  executed  in  an  attempt  to  reconstruct  the 
original  signal  as  accurately  as  possible.  The  inverse  transform  reconstructs  an  approximation  of 
the  original  signal  (Burras,  Gopinath,  and  Guo  1998,  pp.  206-207).  A  simple  block-diagram 
depiction  of  this  process  is  shown  in  Figure  8  that  is  used  in  a  real-world  system. 
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Figure  8.  1-D  Reconstruction  DWT  Filter  with  Quantization 

When  the  inverse  wavelet  transform  is  performed,  loss  of  information  due  to  quantization  errors 
may  cause  the  resulting  decompressed  signal  to  differ  measurably  from  the  original  signal.  This 
difference  may  be  expressed  as  a  mean-squared  error  (MSE)  between  the  original  k-size  signal  v(k) 
and  the  decompressed  k-size  signal  v’(k): 


mse  =  - -X(vM-v'M)2 

^  ’  n 

where 


(?) 


N  =  length  of  vector  v. 

Note  that  MSE  increases  in  proportion  to  the  degree  of  quantization  introduced  into  the  transmitted 
signal. 

For  most  image-processing  applications,  MSE  is  the  figure-of-merit  of  choice  for  quantifying  the 
amount  of  quantization  noise,  which  in  turn  becomes  one  of  the  most  important  factors  that 
quantifies  the  quality  of  the  reconstructed  signal.  In  this  paper,  MSE  is  utilized  to  quantify 
quantization  noise  resulting  of  a  discrete  mapping  of  an  arbitrary  array  of  image  data  points  onto  a 
smaller  size/range  of  image  data  points.  As  mentioned  previously,  quantization  noise  is  one  of  the 
few  non-linear  processes  which  deliberately  discards  signal  information  which  automatically  makes 
it  much  more  difficult  to  reconstruct  the  original  signal.  This  statement  is  worth  repeating  since  the 
population-based  search  properties  of  GAs  have  proven  to  be  particularly  effective  in  identifying 
optimized  solutions  to  various  non-linear  processes. 


5.  RESEARCH  OBJECTIVE:  MINIMIZING  QUANTIZATION  NOISE 

For  image  compression,  the  main  objective  is  to  reduce  the  amount  of  data  to  be  transmitted.  This 
is  especially  true  for  bandwidth-limited  systems.  Unfortunately,  on  the  receiving  end,  the  image 
can  never  be  perfectly  reconstructed  using  standard  wavelet  inverse  transforms.  The  required 
information  is  already  gone.  One  can  only  hope  to  adjust  the  resultant  image  in  some  way  as  to 
minimize  the  MSE. 

As  mentioned  previously,  adaptive  filtering  techniques  are  utilized  for  optimizing  filter  coefficients 
of  some  plant/model  until  the  resultant  output  signal  emulates  the  original  signal  as  close  as 
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possible.  Specifically,  can  Adaptive  Filter  Techniques  compensate  for  information  loss  due  to 
quantization?  This  observation  raises  the  key  question  to  be  addressed  in  this  research: 

Is  it  possible  to  use  a  GA  to  “< adaptively ”  evolve  a  set  of  optimized  g  and  h  coefficients  for  an 
inverse  wavelet  transform,  such  that  the  MSE  of  images  reconstructed  by  this  inverse  transform 
is  significantly  less  than  the  mean  squared  error  of  images  reconstructed  by  the  standard  wavelet 
inverse  transform? 

As  such,  the  purpose  of  this  investigation  is  to  determine  whether  a  GA  (Goldberg  1989)  can  be 
used  to  automatically  optimize  sets  of  wavelet  coefficients  that  outperform  standard  wavelets  for 
such  applications  as  (1-D)  vector  reconstruction  (Bradley,  Brislawn,  and  Hopper  1994)  and  (2-D) 
image  reconstruction  (Saha  2000). 


6.  SENSITIVITY  ANALYSIS  OF  ORTHONORMAL  BASIS  FUNCTIONS 

Before  a  discussion  of  the  actual  GA/Wavelet  modification  experiments,  a  short  demonstration  on 
how  changes  to  some  filter  coefficients  impacts  the  signal  reconstruction  process.  Perfectly 
reconstructed  1-D  signals  or  2-D  images  are  only  possible  due  to  the  properties  of  the  orthonormal 
basis  vectors  used  in  the  DWT.  For  the  1-D  case,  assume  that  a  set  of  vectors  spans  some  vector 
space  such  as  time.  Figure  9  details  how  this  experiment  will  take  place.  The  filter  coefficients  are 
used  for  the  inverse  transform  DWT'1. 


Figure  9.  1-D  Reconstruction  DWT  Filter  with  Quantization 

The  adjustable  arrow  through  the  DWT'1  in  Figure  9  represents  the  percentage  change  to  the 
original  Daub4  inverse  transform  filter  coefficients.  The  Daub4  filter  coefficients  used  for  the 
analysis  and  synthesis  functions  are  shown  in  Table  2. 
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Table  2.  Original  Discrete  Daub4  Wavelets 


Filter 

# 

Coefficients 

Change 

(%) 

hi 

-0.12941 

0.22414 

0.83652 

0.48296 

0 

gl 

-0.4829 

0.83652 

-0.2241 

-0.12941 

0 

h2 

0.48296 

0.83652 

0.22414 

-0.12941 

0 

g2 

-0.12941 

-0.22414 

0.83652 

-0.48296 

0 

The  hi  and  gl coefficients  are  used  respectively  in  the  H(z)  and  G(z)  forward  transform  blocks 
shown  previously  in  Figure  9.  The  h2  and  g2  coefficients  are  used  respectively  in  the  H(z)1  and 
G(z)'1  reverse  transform  blocks  shown  previously  in  Figure  9. 

A  demonstration  of  the  power  of  perfect  reconstruction  with  0%  change  to  the  DWT"1  coefficients 
will  now  be  demonstrated.  The  test  input  signal  “v(k)”  will  be  a  1-D  sine-wave  in  this  case.  The 
v(k)  signal  is  transformed  using  the  “hi  and  gl”  filter  coefficients  shown  in  Table  2  and  then 
downsampled  as  shown  previously  in  Figure  9.  The  resultant  signal  is  then  upsampled  and  inverse- 
transformed  using  the  “h2  and  g2”  coefficients  shown  in  Table  2  above.  The  results  of  this  process 
are  shown  in  Figure  10. 


Figure  10.  1-D  Sine  Wave  Perfect  Reconstruction  Daub4  Filter  without  Quantization 

In  Figure  10,  the  input  signal’s  symbol  is  “v(k)”  and  the  output  symbol  stands  is  “v(k)”\  The 
“e(k)”  in  this  case  is  simply  the  magnitude  error  difference  between  the  two  signals  at  a  specific 
point  in  time.  Specifically  “e(k)”  is  defined  by: 

e(k )  =  v(k )  -  v(k)'  (8) 
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Now,  as  a  second  experiment,  the  h2  and  g2  filter  coefficients  are  adjusted  manually  by  20%  each 
as  shown  in  Table  3  below.  The  choice  of  20%  is  arbitrary  and  is  for  illustrative  purposes  only. 


Table  3.  20%  Change  in  Daub4  Wavelets 


Filter 

# 

Coefficients 

Change 

(%) 

hi 

-0.12941 

0.22414 

0.83652 

0.48296 

0 

gl 

-0.4829 

0.83652 

-0.2241 

-0.12941 

0 

h2 

0.57956 

1.0038 

0.26897 

-0.10353 

20 

g2 

-0.10353 

-0.17932 

1.0038 

-0.38637 

20 

The  results  of  the  same  reconstruction  process  is  now  shown  in  Figure  11. 


Figure  11.  1-D  Sine  Wave  Daub4  Filter  with  20%  Change  of  h2  &  g2  DWT  Coefficients 

The  same  process  will  now  be  repeated  as  another  example  but  now  with  images.  Figure  12 
demonstrates  the  power  of  perfect  reconstruction  for  images,  with  0%  change  to  the  DWT'1 
coefficients  on  the  standard  256X256  “Barb”  image.  The  Daub4  filter  coefficients  used  for  the  sine 
wave  analysis  and  synthesis  functions  are  the  same  as  what  is  shown  in  Table  2. 
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Figure  12.  2-D  “Barb”  Image  using  Daub4  Filter  with  0%  Change  of  h2  &  g2  DWT  Coefficients 


Figure  13  shows  the  same  image  reconstructed  with  a  20%  change  in  the  h2  and  g2  filter 
coefficients.  The  coefficients  are  shown  in  Table  3. 


Figure  13.  2-D  “Barb”  Image  with  a  20%  Change  in  Daub4  h2  &  g2  Wavelets 


Note  that  there  are  a  vast  number  of  ways  to  adjust  these  filter  coefficients.  However,  for 
demonstration  purposes  only  it  was  decided  to  only  adjust  the  filter  coefficients  in  one  direction 
(i.e.,  positive)  with  the  same  amount  of  magnitude  (i.e.,  20%).  The  main  purpose  of  this  section  is 
to  give  the  reader  a  flavor  of  what  manually  adjusting  the  filter  coefficients  of  the  DWT  can  do. 
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This  short  experiment  highlights  the  fact  that,  even  with  the  modification  of  the  “h2”  and  “g2”  filter 
coefficients,  the  resultant  reconstructed  1-D  and  2-D  information  can  still  be  recognizable  by  the 
human  eye.  As  a  result,  one  may  state  the  coefficients  are  quite  “robust”  which  make  them  a  very 
suitable  to  be  used  as  variables  in  this  adaptive  information  processing  system. 


7.  THE  GENETIC  ALGORITHM 

Optimization  is  a  process  by  which  the  object  is  to  “enhance”  some  parametric  output  of  a  system. 
Genetic  Algorithms  are  one  such  process  that  uses  evolution  as  its  basic  model  to  formulate  various 
stochastic  rules.  The  GA  developed  for  this  research  effort  closely  follows  a  standard  GA  (Holland 
1975),  employing  fitness-driven  selection,  crossover,  and  mutation  operators  to  create  each  new 
generation  of  candidate  solutions.  The  main  components  to  most  standard  GAs  are  shown  below  in 
Figure  14  below. 


Figure  14.  Diagram  of  a  basic  GA 
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The  initial  population  is  usually  generated  by  some  random  number  generator  algorithm  to  create  an 
initial  array  of  populations.  The  selection,  mating,  and  crossover  processes  are  then  used  to  decide 
which  parts  (i.e.,  genes)  of  which  parents,  from  the  current  population,  are  to  be  used  to  create  the 
new  genes  of  the  offspring.  However,  sometimes  the  GA  may  sometimes  find  itself  “stuck”  in  a 
local  minimum.  In  situations  like  this  “mutation”  is  utilized  to  “guess”  a  new  starting  point  to  kick- 
start  the  whole  process  again.  And  finally  the  fitness  function  is  use  to  select  the  “optimum” 
children  to  be  used  to  regenerate  the  whole  process  all  over  again  only  if  some  predetermined 
optimum  point  has  not  been  reached  yet. 

In  the  previous  section,  it  is  demonstrated  that  by  “manually”  adjusting  the  DWT1  “g2”  and  “h2” 
affected  the  resultant  reconstructed  data.  Using  a  GA,  this  step  will  now  be  accomplished 
“automatically”  in  the  hopes  of  minimizing  the  quantization  noise  for  images.  Prior  to  this  research, 
it  was  not  known  whether  any  academic,  commercial,  or  military  organization  had  attempted  to  use 
GAs  to  derive  (or  select  and  then  optimize)  alternative  sets  of  “evolved”  wavelet  filter  coefficients 
that  perform  better  than  standard  wavelets  for  particular  applications.  Instead,  memory-consuming 
look-up  tables  have  historically  been  utilized  by  various  DSP  embedded  processors.  What  is 
needed,  especially  in  the  military  operations  field,  is  a  system  that  derives  particular  optimized 
filters  in  real  time  for  a  particular  mission  mode  of  operation  (e.g.,  specifications)  and  background 
clutter  (e.g.,  raw  data).  Such  a  system  potentially  negates  the  necessity  of  using  large  look-up  tables 
stored  in  RAM  or  ROM.  For  this  effort,  the  GA  was  utilized  to  “tweak”  the  least  significant  bits  of 
preselected  wavelet  filter  coefficients.  The  purpose  of  the  fitness  function  was  to  lower,  if  possible, 
the  quantization  noise  to  some  predetermined  acceptable  level.  This  new  process  is  depicted  in 
Figure  15  which  shows  the  top-level  key  components  of  the  GA/DWT  solution. 


Figure  15.  Basic  GA  Wavelet  Optimization  Experiment 
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Since  encoding  and  decoding  of  the  transmitted  signal  is  not  the  mail  problem  with  image 
compression,  neither  of  their  algorithms  will  be  utilized  in  these  experiments.  Only  the  quantization 
and  DWT  algorithms  will  use  to  conduct  all  experiments. 


Figure  16  is  a  detailed  block  diagram  of  the  whole  GA/DWT  optimization  process.  It  provides  a 
detailed  illustration  of  the  evolutionary  process  it  embodies. 


Figure  16.  Detailed  GA  Wavelet  Optimization  Experiment. 

Specifically,  the  above  GA  executes  the  following  steps: 

1.  The  initial  population  (generation  0)  is  represented  by  an  array  of  M  candidate  solutions. 
Each  candidate  solution  consists  of  a  set  of  g2  and  h2  values  (as  defined  for  the  inverse 
wavelet  transform).  The  GA  begins  by  initializing  the  g2  and  h2  values  for  each  candidate 
solution  to  the  values  from  the  selected  standard  wavelet.  Candidate  solution  0  of 
generation  0  remains  unchanged.  For  each  of  the  remaining  M- 1  candidate  solutions,  each 
of  the  g2  and  h2  values  is  mutated  by  a  small,  random  multiplicative  factor  (e.g.,  0.99  or 
1.02).  Thus,  generation  0  is  initially  seeded  with  of  one  exact  copy  and  M- 1  random 
perturbations  of  the  selected  standard  wavelet  (Garza  and  Maher  1999). 

2.  Next,  the  GA  runs  for  a  fixed  number  of  generations  (G). 

a)  For  each  generation,  the  fitness  of  each  candidate  solution  from  current  population  is 
evaluated.  For  the  vector  and  image  data  used  in  this  investigation,  fitness  equals  the 
summed  magnitude  of  error  between  the  original  image  and  the  image  reconstructed 
using  the  wavelet  coefficients  that  comprise  the  candidate  solution. 
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b)  Tournament  selection  (Miller  and  Goldberg  1995)  is  then  used  to  populate  the  next 
generation.  With  tournament  selection,  a  predetermined  number  of  individuals  are 
randomly  selected  from  the  current  generation,  and  the  candidate  solution  with  the  best 
fitness  value  is  copied  into  the  next  generation.  This  process  is  repeated  M  times  in 
order  to  populate  the  next  generation  with  M  candidate  solutions. 

c)  Next,  crossover  is  performed.  Crossover  pairs  each  member  of  the  next  generation  with  a 
second  randomly  selected  individual.  For  this  investigation,  two  crossover  points  (one 
for  the  g2  vector  and  one  for  the  h2  vector)  are  identified,  and  the  values  at  and  below 
each  crossover  point  are  exchanged.  Thus,  the  genetic  composition  of  each  child 
individual  resulting  from  the  crossover  operation  typically  differs  from  that  of  both  of 
the  selected  parent  individuals. 

d)  Next,  mutation  is  randomly  applied  to  a  predetermined  and  small  percentage  of  the 
coefficients  in  each  of  the  candidate  solutions  in  the  next  generation.  For  this  problem, 
mutation  consists  of  modifying  a  given  coefficient  by  a  small  multiplicative  factor, 
according  to  a  selected  Gaussian  distribution.  Samples  of  this  are  shown  in  Table  4 
below. 


Table  4.  20%  Change  in  Daub4  Wavelets 


Factor 

Mutation  % 

Probability 

1.01 

+1% 

5% 

1.02 

+2% 

3% 

1.03 

+3% 

1% 

0.97 

-3% 

1% 

0.98 

-2% 

3% 

0.99 

-1% 

5% 

-1.0 

{sign  change) 

1% 

e)  Finally,  the  next  generation  simply  replaces  the  current  generation. 

3.  After  repeating  step  2  for  G  generations,  the  individual  with  the  best  fitness  of  the  entire  run 
is  declared  the  solution.  For  this  study,  the  best-of-run  solution  defines  the  set  of  g2  and  h2 
coefficients  that  minimizes  the  total  error  in  the  reconstructed  signal. 

By  forcing  generation  0  to  include  a  verbatim  copy  of  the  standard  wavelet,  it  guarantees  that  the 
solution  evolved  by  the  GA  can  do  no  worse  than  the  standard  wavelet.  By  seeding  the  initial 
population  with  randomly  mutated  copies  of  the  selected  standard  wavelet,  the  evolutionary  process 
becomes  biased  such  that  it  focuses  upon  the  immediately  adjacent  solution  space.  In  addition,  this 
GA  can  be  modified  such  that  the  best  solution  from  the  current  generation  is  copied  into  the  next 
generation;  this  process  guarantees  that  the  best-of-generation  fitness  value  will  not  worsen  from 
one  generation  to  the  next. 


8.  RESULTS 

Twelve  tests  were  performed  using  2-D  image  data.  Each  test  was  characterized  by  three  factors: 
1.  The  quantization  factor:  These  tests  used  quantization  values  of  32  and  64. 
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2.  The  2-D  data  being  reconstructed:  Several  images  commonly  appearing  in  the  wavelet 
literature  were  used,  including  “Airplane”,  “Baboon”,  “Barb”,  “Boat”,  “Couple”,  “Fruits”, 
“Goldhill”,  “Lenna”,  “Park”,  “Peppers”,  “Susie”,  and  “Zelda”. 

3.  The  selected  standard  wavelet:  Wavelets  used  in  these  tests  included  the  Daub4 
(Daubechies  1988),  2/6  (Villasenor,  Belzer,  and  Liao  1995),  and  5/3  (Lin  1999)  wavelets. 

Each  test  consisted  of  two  runs.  The  first  run  used  the  unmodified  standard  wavelet  to  reconstruct  a 
selected  image.  The  second  run  used  a  GA  to  evolve  a  new  set  of  wavelet  coefficients  for  a 
similarly  structured  wavelet.  The  result  of  each  test  compared  the  MSE  and  PSNR  (Vetterli  and 
Kovacevic  1995)  of  the  evolved  wavelet  to  the  standard  wavelet.  These  results  compare  the 
evolved  coefficients  to  the  standard  coefficients,  and  state  the  percentage  change  in  the  magnitude 
of  each  coefficient. 

TEST  RESULTS  USING  DAUB4:  The  coefficients  for  the  standard  Daub4  inverse  wavelet 
transform  are  shown  below  and  their  plots  are  shown  in  Figure  16  below. 

h2  =  {0.482962913,  0.836516304,  0.224143868,  -0.129409523} 
g2  =  {-0.129409523,  -0.224143868,  0.836516304,  -0.482962913} 


Figure  16.  Standard  Daub4  Scaling  Function  and  Wavelet 
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TEST  1:  Quantization  =  32,  Image  =  “Barb” 

Daub4:  MSE  =  96.886522928  PSNR  =  8.268169906 

Evolved:  MSE  =  92.904186248  PSNR  =  28.450450771 

Percentage  Change:  MSE  =  -4. 1 103%%  PSNR  =  +0.6648% 

Evolved  coefficients:  h2’  =  {0.46824367,  0.81126518,  0.23985918,  -0.10537719} 
g2’  =  {-0.17815653,  -0.18627964,  0.78603730,  -0.44936688} 
Percentage  change:  Ah2  =  {-3.048%,  -3.019%,  +7.01 1%,  -18.571%} 

Ag2  =  {+37.669%,  -16.893%,  -6.034%,  -6.034%} 


Figure  17.  Resultant  Daub4  Scaling  Function  and  Wavelet  with  Coefficients  Evolved  Using  “Barb” 

Image  (Quantization  =  32) 
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TEST  2:  Quantization  =  32,  Image  =  “Boat” 

Daub4:  MSE  =  52.520229339  PSNR  =  30.927537470 

Evolved:  MSE  =  49.920497894  PSNR  =  31.148014526 

Percentage  Change:  MSE  =  -4.9500%  PSNR  =  +0.7 129% 

Evolved  coefficients:  h2’  =  {0.47813328,  0.80323285,  0.22855950,-0.09641626} 
g2’  =  {-0.20371400,  -0.11414577,  0.76260869,-0.38425025} 
Percentage  Change:  Ah2  =  {-1.000%,  -3.979%,  +1.970%,  -29.495%} 

Ag2  =  {+57.418%,  -49.074%,  -8.835%,  -20.439%} 


Figure  18.  Resultant  Daub4  Scaling  Function  and  Wavelet  with  Coefficients  Evolved  Using  “Boat” 

Image  (Quantization  =  32) 
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TEST  3:  Quantization  =  64,  Image  =  “Couple” 

Daub4:  MSE  =  155.96321105  PSNR  =  26.200581927 

Evolved:  MSE  =  144.96800359  PSNR  =  26.518082027 

Percentage  Change:  MSE  =  -7.0499%  PSNR  =  +1.21 18% 

Evolved  Coefficients:  h2’  =  {0.46393225,  0.78701072,  0.24254917,-0.07969529} 
g2’  =  {-0.25739530,  -0.07244071,  0.71684818,  -0.34936503} 
Percentage  Change:  Ah2  =  {-3.940%,  -2.990%,  +8.21 1%,  -38.416%} 

Ag2  =  {+98.900%,  -67.681%,  -14.306%,  -27.662%} 


Figure  19.  Resultant  Daub4  Scaling  Function  and  Wavelet  with  Coefficients  Evolved  Using 

“Couple”  Image  (Quantization  =  64) 
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TEST  4:  Quantization  =  64,  Image  =  “Fruits” 

Daub4:  MSE  =  115.80363591  PSNR  =  27.493581656 

Evolved:  MSE  =  1 10.45684432  PSNR  =  27.698877292 

Percentage  Change:  MSE  =-4.6171%  PSNR  = +0.7467% 

Evolved  Coefficients:  h2’  =  {0.45006297,  0.81150696,  0.25752134,  -0.10514879} 
g2’  =  {-0.15772549,  -0.13019651,  0.71494264,-0.38877090} 
Percentage  Change:  Ah2  =  {-6.812%,  -2.990%,  +14.891%,  -18.747%} 

Ag2  =  {+21.881%,  -41.914%,  -14.533%,  -19.503%} 


“Fruits”  Image  (Quantization  =  64) 
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SUMMARY  OF  RESULTS  OF  TESTS1-4  USING  DAUB4  WAVELETS: 


Average  Improvement,  MSE:  5.181 8% 

Average  Improvement,  PSNR:  0.8341% 

These  results  indicate  that  the  GA  was  capable  of  evolving  a  set  of  real-valued  coefficients  that 
consistently  outperformed  a  similarly  structured  Daub4  wavelet  using  standard  wavelet  coefficients. 
As  a  sample  of  one  of  these  results,  images  before  and  after  the  transformation  conducted  for 
“TEST3”  are  shown  in  Figures  21-23.  Close  inspection  of  Figure  23  reveals  minor  improvements 
over  the  image  shown  in  Figure  22  which  was  reconstructed  via  the  standard  Daub4  wavelet. 


Figure  21.  The  Original  “Couple”  Image 


Figure  22.  “Couple”  Reconstructed  via  the  Standard  Daub4  Wavelet  (Quantization  =  64) 
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Figure  23.  “Couple”  Reconstructed  via  an  Evolved  Wavelet  (Quantization  =  64). 
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RESULTS  USING  2/6  (TSi  MOTHER  WAVELETS:  The  coefficients  for  the  standard  TS 
inverse  wavelet  transform  are  shown  below  and  their  plots  are  shown  in  Figure  24. 

h2  =  {-0.088388,  0.088388,  0.707107,  0.707107,  0.088388,  -0.088388} 
g2  =  {-0.70710678,  0.70710678} 


Figure  24.  Standard  TS  Scaling  Function  and  Wavelet 
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TEST  5:  Quantization  =  32,  Image  =  “Goldhill” 

TS:  MSE  =  55.693927764  PSNR  =  30.672725 136 

Evolved:  MSE  =  52.678527832  PSNR  =  30.914467313 

Percentage  Change:  MSE  =-5.4142%  PSNR  = +0.7881% 

Evolved  Coefficients:  h2’  =  {-0.07478096,  0.11289088,  0.68569361, 
0.66566063,  0.09555358,  -0.07139243} 

g2’  =  {-0.61834085,  0.61191382} 

Percentage  Change:  Ah2  =  {-15.395%,  +27.075%,  -3.028%, 

-5.861%,  +8.107%,  -19.228%} 
Ag2  =  {-12.553%,  -12.331%} 


Figure  25.  Resultant  TS  Scaling  Function  and  Wavelet  with  Coefficients  Evolved  Using  “Goldhill” 

Image  (Quantization  =  32) 
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TEST  6:  Quantization  =  32,  Image  =  “Park” 

TS:  MSE  =  68.487875620  PSNR  =  29.774666655 

Evolved:  MSE  =  65.506455739  PSNR  =  29.967962585 

Percentage  Change:  MSE  =-4.3532%  PSNR  =+0.6492% 

Evolved  Coefficients:  h2’  =  {-0.07336637,  0.11436371,  0.67917624, 

0.67238448,  0.10247711,  -0.07791956} 
g2’  =  {-0.65717215,  0.66505474} 

Percentage  Change:  Ah2  =  {-16.995%,  +29.388%,  -3.950%, 

-  4.911%,  +15.940%,  -11.844%} 

Ag2  =  {-7.062%,  -5.947%} 


Figure  26.  Resultant  TS  Scaling  Function  and  Wavelet  with  Coefficients  Evolved  Using  “Park” 

Image  (Quantization  =  32) 


28 


TEST  7:  Quantization  =  64,  Image  =  “Peppers” 

TS:  MSE=  124.52128283  PSNR  =  27.178367746 

Evolved:  MSE  =  1 12.26459054  PSNR  =  27.628375641 

Percentage  Change:  MSE  =  -9.843 1%  PSNR  =  +1 .6558% 

Evolved  Coefficients:  h2’  =  {-0.06290282,  0.12320526,  0.66566063, 

0.63949490,  0.10331971,  -0.05645800} 
g2’  =  {-0.67870071,  0.78100922} 

Percentage  Change:  Ah2  =  {-28.946%,  +39.391%,  -5.861%, 

-9.562%,  +16.893%,  -36.125%} 

Ag2  =  {-4.107%,  +10.451%} 


Figure  27.  Resultant  TS  Scaling  Function  and  Wavelet  with  Coefficients  Evolved  Using  “Peppers” 

Image  (Quantization  =  64) 
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TEST  8:  Quantization  =  64,  Image  =  “Susie” 

TS:  MSE  =  133.22735341  PSNR  =  26.884869602 

Evolved:  MSE  =  1 18.87997182  PSNR  =  27.379716673 

Percentage  Change:  MSE  =-10.7691%  PSNR  =  +1.8406% 

Evolved  Coefficients:  h2’  =  {-0.06581485,  0.12938192,  0.66566063, 
0.63309995,  0.10846400,  -0.05439425} 
g2’  =  {-0.61788622,  0.72802296} 

Percentage  Change:  Ah2  =  {-25.539%,  +46.380%,  -5.861%, 

-10.466%,  +22.713%,  -38.460%} 
Ag2  =  {-12.618%,  +2.958%} 


Image  (Quantization  =  64) 
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SUMMARY  OF  RESULTS  OF  TESTS  5-8  USING  TS  WAVELETS: 


Average  Improvement,  MSE:  7.5949% 

Average  Improvement,  PSNR:  1 .2334% 

These  results  indicate  that  our  GA  was  capable  of  evolving  a  set  of  real-valued  coefficients  that 
consistently  outperformed  a  similarly  structured  TS  wavelet  using  standard  wavelet  coefficients. 
Improvements  in  both  the  MSE  and  the  PSNR  for  the  TS  wavelet  were  consistently  greater  than  the 
corresponding  improvement  for  the  Daub4  wavelet  seen  in  the  first  four  tests. 

The  sample  image  results  for  “TEST  8”  are  shown  in  Figures  29-31.  As  can  be  seen,  slight 
improvements  in  shading  and  detail  are  visible  in  Figure  31  as  compared  the  image  reconstructed 
using  the  standard  TS  wavelet  shown  in  Figure  30. 


Figure  29.  The  Original  “Susie”  Image 


Figure  30. 


‘Susie’ 


Reconstructed  via  the  Standard  TS  Wavelet  (Quantization  =  64). 
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Figure  31.  “Susie”  Reconstructed  via  an  Evolved  Wavelet  (Quantization  =  64). 


32 


RESULTS  USING  5/3  (FT)  MOTHER  WAVELETS:  The  coefficients  for  the  standard  FT 
inverse  wavelet  transform  are  shown  below  along  with  their  corresponding  plots  in  Figure  32. 

h2  =  {0.353553,  0.70710678,  0.353553} 
g2  =  {-0.176777,  -0.353553,  1.060660,  -0.353553,  -0.176777} 


Figure  32.  Standard  FT  Scaling  Function  and  Wavelet 
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TEST  9:  Quantization  =  32,  Image  =  “Airplane” 

FT:  MSE  =  35.743078867  PSNR  =  32.598884014 

Evolved:  MSE  =  35.453097025  PSNR  =  32.634261816 

Percentage  Change:  MSE  =  -0.81 13%  PSNR  = +0.1085% 

Evolved  Coefficients:  h2’  =  {0.35708853,  0.70703607,  0.35001747} 
g2’  =  {-0.22535310,  -0.34301712,  1.09116094, 

-0.35623248,  -0.22755308} 

Percentage  Change:  Ah2  =  {+1.000%,  -0.010%,  -1.000%} 

Ag2  =  {+27.479%,  -2.980%,  +2.875%, 

+0.758%,  +28.723%} 


Figure  33.  Resultant  FT  Scaling  Function  and  Wavelet  with  Coefficients  Evolved  Using  “Airplane” 

Image  (Quantization  =  32) 
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TEST  10:  Quantization  =  32,  Image  =  “Baboon” 

FT:  MSE=  113.09576543  PSNR  =  27.596340166 

Evolved:  MSE  =  111.85707219  PSNR  =  27.644169131 

Percentage  Change:  MSE  =  -1.0953%  PSNR  = +0.1733% 

Evolved  Coefficients:  h2’  =  {0.35001747,  0.70682398,  0.35708853} 
g2’  =  {-0.20626139,  -0.31293066,  0.99595579 

-0.31273186,  -0.20463438} 

Percentage  Change:  Ah2  =  {-1.000%,  -0.040%,  1.000%} 

Ag2  =  {+16.679%,  -1 1.490%,  -6.100%, 

-11.546%, +15.758%} 


Figure  34.  Resultant  FT  Scaling  Function  and  Wavelet  with  Coefficients  Evolved  Using  “Baboon” 

Image  (Quantization  =  32) 
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TEST  11:  Quantization  =  64,  Image  =  “Lenna” 

FT:  MSE=  140.01809946  PSNR  =  26.668961823 

Evolved:  MSE  =  139.92291514  PSNR  =  26.671915162 

Percentage  Change:  MSE  =  -0.0680%  PSNR  =  +0.01 11% 

Evolved  Coefficients:  h2’=  {0.34627481,  0.70689467,  0.36051517} 
g2’  =  {-0.26215020,  -0.26897337,  0.77367288, 
-0.33877071,-0.21314989} 

Percentage  Change:  Ah2  =  {-2.059%,  -0.030%,  +1.969%} 

Ag2  =  {+48.294%,  -23.923%,  -27.057%, 

-4.181%,  +20.576%} 


Figure  35.  Resultant  FT  Scaling  Function  and  Wavelet  with  Coefficients  Evolved  Using  “Lenna” 

Image  (Quantization  =  64) 


36 


TEST  12:  Quantization  =  64,  Image  =  “Zelda” 

FT:  MSE=  111.79914855  PSNR  =  27.646418648 

Evolved:  MSE  =  1 10.98122024  PSNR  =  27.678308652 

Percentage  Change:  MSE  =  -0.731 6%  PSNR  =  +0.1153% 

Evolved  Coefficients:  h2’  =  {0.33251660,  0.70668262,  0.37494296} 
g2’  =  {-0.18713481,  -0.36715097,  0.85353342, 

-0.28083948,-0.44643196} 

Percentage  Change:  Ah2  =  {-5.950%,  -0.060%,  +6.050%} 

Ag2  =  {+5.859%,  +3.846%,  -19.528%, 

-20.567%, +152.540%} 


Image  (Quantization  =  64) 
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SUMMARY  OF  RESULTS  OF  TESTS  9-12  USING  FT  WAVELETS: 


Average  Improvement,  MSE:  0.6766% 

Average  Improvement,  PSNR:  0. 1 02 1  % 

These  results  indicate  that  the  GA  is  again  capable  of  evolving  a  set  of  real- valued  coefficients. 
Unfortunately  in  this  case  the  percentage  of  improvement  is  much  smaller  than  previously  noted  for 
the  Daub4  and  2/6  wavelets. 

Sample  image  results  for  “TEST  10”  are  shown  in  Figures  37-39.  As  can  be  seen,  differences 
between  Figures  38  and  39  are  too  small  to  be  perceived  by  the  naked  eye. 


^ _ ' _ _ _ _ _ _ i _ 

Figure  37.  The  Original  “Baboon”  Image 


Figure  38.  “Baboon”  Reconstructed  via  the  Standard  FT  Wavelet  (Quantization  =  32). 
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Figure  39.  “Baboon”  Reconstructed  via  an  Evolved  Wavelet  (Quantization  =  32). 

OVERALL  RESULTS  (ALL  12  TESTS): 

Average  Improvement,  MSE:  4.4844% 

Average  Improvement,  PSNR:  0.7232% 

9.  CONCLUSION 

Prior  to  this  study,  it  was  believed  that  not  too  many  active  wavelet  researchers  would  have  believed 
it  possible  to  systematically  modify  the  fdter  coefficients  of  standard,  “off-the-shelf’  wavelets  in  the 
hopes  of  reducing  MSE  and  improving  PSNR.  However,  the  results  of  this  effort  conclusively 
illustrate  that  wavelet  fdter  coefficients  can  be  optimized  via  a  GA.  This  process  improves  upon  the 
performance  of  similarly  structured  standard  wavelets  (such  as  the  Daub4,  TS,  and  FT  wavelets 
used  in  these  tests)  for  lossy  image  reconstruction  applications. 


10.  APPLICATIONS  OF  THE  NEW  TECHNOLOGY  AND  FUTURE  RESEARCH 

Although  most  of  the  incremental  improvements  summarized  in  the  document  are  at  best  barely 
detectable  to  the  naked  eye,  they  nevertheless  appear  to  have  established  a  basic  method  that  may 
ultimately  lead  to  significant  enhancement  of  modern  image  reconstruction  technology.  The 
approach  established  by  this  investigation  promises  improvement  upon  state  of  the  art  wavelet 
decompression  techniques  for  a  wide  range  of  applications,  including  audio  denoising  (Schremmer, 
Haenselmann,  and  Bomers  2001),  sea  clutter  noise  reduction  for  radar  proximity  fusing  (Noel  and 
Szu  1998),  signal  compression  (Saito  1994),  object  detection  (Zhu  and  Schwartz  2002),  fingerprint 
compression  (Bradley,  Brislawn,  and  Hopper  1993),  image  denoising  (Chang,  Yu,  and  Vetterli 
1998)  ,  image  enhancement  (Wang,  Wu,  Castleman,  and  Xiong  2001),  image  recognition  (Schilling, 
Cosman,  and  Berry  1998),  and  speech  recognition  (Long  and  Datta  1998). 
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This  research  demonstrates  that  a  GA  can  evolve  coefficients  for  the  inverse  wavelet  transform  that 
reduced  MSE  in  comparison  to  a  standard  wavelet  transform  of  the  same  image.  The  question  of 
whether  a  single  wavelet  could  be  evolved  that  outperforms  standard  wavelets  for  a  specific  class  of 
images  (e.g.,  satellite  images),  or  even  for  all  possible  images,  has  yet  to  be  addressed.  Future 
efforts  will  attempt  to  evolve  a  “super  wavelet”  capable  of  producing  higher  fidelity  reconstructions 
of  transformed,  quantized  images  than  any  previously  identified  wavelet. 

This  research  evolved  coefficients  of  inverse  wavelet  transforms  that  minimize  the  total  MSE  in 
reconstructed  images  that  had  originally  been  (forward)  transformed  by  wavelets  utilizing  standard 
coefficients.  It  is  possible,  however,  that  the  overall  performance  of  our  evolved  wavelet 
reconstruction  system  could  be  further  improved  by  simultaneously  evolving  coefficients  for  the 
forward  wavelet  transform.  In  effect,  this  effort  would  attempt  to  evolve  sets  of  coefficients  for 
matching  forward  and  inverse  transforms  under  conditions  subject  to  quantization  error.  Future 
research  should  therefore  investigate  the  possibility  of  simultaneously  evolving  coefficients  for  both 
the  forward  and  inverse  wavelet  transform. 

The  GA  developed  for  this  study  uses  an  initial  population  that  is  created  by  randomly  mutating 
copies  of  a  selected  standard  wavelet.  This  process  biased  the  search,  causing  the  GA  to  focus 
primarily  upon  the  solution  space  immediately  adjacent  to  the  standard  wavelet.  Hence,  the  evolved 
wavelet  is  typically  very  similar  to  the  standard  wavelet.  It  is  as  yet  unknown  whether  the 
evolutionary  process  could  locate  optimized  solutions  in  entirely  new  and  previously  unexploited 
neighborhoods  of  the  solution  space.  Future  work  will  therefore  include  attempts  to  evolve 
wavelets  without  imparting  any  bias  to  the  search  process. 

Noise  (Fante  1988)  refers  to  the  introduction  of  any  undesired  change  in  the  information  content  of 
a  signal  during  transmission.  Thresholding  (the  removal  of  wavelet  coefficients  whose  magnitude 
is  less  than  a  predetermined  noise  threshold)  is  often  used  to  eliminate  random  noise  (Donoho  and 
Johnstone  1994).  Various  wavelet-based  methods  have  also  been  shown  to  be  highly  effective 
(Donoho  1993).  In  many  situations,  however,  a  transformed  signal  is  likely  to  contain  several  low- 
energy  values  that  may  make  an  important  contribution  to  an  accurate  reconstruction  of  the  original 
signal.  If  these  values  lie  below  the  noise  threshold,  denoising  will  remove  them  from  the 
transformed  signal,  and  subsequent  reconstruction  may  produce  an  unsatisfactory  approximation  of 
the  original  signal  (Malfait  1996).  Although  the  current  research  effort  focused  primarily  upon  the 
elimination  of  error  due  to  the  quantization  process,  future  research  may  expand  upon  our  results  by 
developing  novel  approaches  to  noise  reduction  in  signal  and  image  processing  applications. 

The  JPEG  2000  image  compression  standard  (Charrier,  Cruz,  and  Larsson  1999)  uses  wavelet 
technology  to  compress  and  subsequently  reconstruct  images.  It  is  possible  that  the  wavelets 
optimized  during  this  study  could  replace  standard  wavelets  for  lossy  image  compression 
applications  in  the  next  JPEG  release.  Future  research  should  specifically  address  the  JPEG 
standard  and  identify  scenarios  in  which  evolved  wavelets  could  potentially  outperform  standard 
JPEG  wavelets.  A  similar  approach  may  also  be  applied  to  other  wavelet-based  standards. 


40 


11.  ACKNOWLEDGMENT 


This  work  was  sponsored  by  the  AFOSR/NM  Optimization  and  Discrete  Mathematics  office.  The 
authors  of  this  paper  would  like  to  thank  Lt.  Col.  Juan  Vasquez,  the  AFOSR/NM  program 
manager,  for  his  guidance  and  support  during  this  effort. 


12.  BIBLIOGRAPHY 

Bradley,  J.  N.,  C.  M.  Brislawn,  and  T.  Hopper  1993.  The  FBI  Wavelet-Scalar  Quantization 
Standard  for  Grey-Scale  Fingerprint  Compression,  Visual  Information  Processing  II,  Vol.  1961: 
293-304,  Proceedings  of  the  SPIE,  Orlando,  FL. 

Burras,  C.  S.,  R.  A.  Gopinath,  and  H.  Guo  1998.  Introduction  to  Wavelets  and  Wavelet  Transforms: 
A  Primer,  Prentice-Hall. 

Chang,  S.  1995.  Image  Interpolation  Using  Wavelet-Based  Edge  Enhancement  and  Texture 
Analysis,  Master’s  Thesis,  University  of  California  at  Berkeley. 

Chang,  G.,  B.  Yu,  and  M.  Vetterli  1998.  Spatially  adaptive  wavelet  thresholding  based  on  context 
modeling  for  image  denoising,  IEEE  Transactions  on  Image  Processing. 

Charrier,  M.,  D.  S.  Cruz,  and  M.  Larsson  1999.  JPEG2000,  the  Next  Millennium  Compression 
Standard  for  Still  Images,  IEEE  International  Conference  on  Multimedia  Computing  and  Systems, 

1:  131-132. 

Cooley,  J.  W.  and  J.  W.  Tukey  1965.  An  Algorithm  for  the  machine  calculation  of  complex  Fourier 
series,  Mathematics  of  Computation,  19:  297-301. 

Daubechies,  I.  1988.  Orthonormal  Bases  of  Compactly  Supported  Wavelets,  Communications  of 
Pure  and  Applied  Mathematics,  41 :909-996. 

Daubechies,  I.  1992.  Ten  Lectures  on  Wavelets,  SIAM. 

Davis,  G.  M.  and  A.  Nosratinia  1998.  Wavelet-based  Image  Coding:  An  Overview,  Applied  and 
Computational  Control,  Signals,  and  Circuits,  1(1). 

Donoho,  D.  1993.  Nonlinear  Wavelet  Methods  for  Recovery  of  Signals,  Densities,  and  Spectra  from 
Indirect  and  Noisy  Data,  in  Daubechies,  I.  (Ed.),  Different  Perspectives  on  Wavelets,  AMS. 

Donoho,  D.  and  I.  Johnstone  1994.  Ideal  Spatial  Adaptation  via  Wavelet  Shrinkage,  Biometrika,  81: 
425-455. 

Fante,  R.L.,  1988.  Signal  Analysis  and  Estimation:  an  Introduction,  John  Wiley. 

Garza,  A.  G.  and  M.  L.  Maher  1999.  An  Evolutionary  Approach  to  Case  Adaptation,  Third 
International  Conference  on  Case-Based  Reasoning  (ICCBR  ’99),  Springer- Yerlag. 


41 


Goldberg,  D.  E.  1989.  Genetic  Algorithms  in  Search,  Optimization,  and  Machine  Learning, 
Addison-Wesley. 

Graps,  A.  1995.  An  Introduction  to  Wavelets,  IEEE  Computational  Science  and  Engineering,  2(2), 
IEEE. 

Haar,  A.  1910.  Zur  Theorie  der  orthogonalen  Funktionensysteme,  Mathematische  Annalen,  69:  331- 
371. 

Holland,  J.  1975.  Adaptation  in  Natural  and  Artificial  Systems,  University  of  Michigan  Press. 

Lin,  C.  1999.  Time  and  Space  Efficient  Wavelet  Transform  for  Real-Time  Applications,  doctoral 
dissertation,  The  Ohio  State  University,  161-2. 

Long,  C.  J.  and  S.  Datta  1998.  Discriminant  Wavelet  Basis  Construction  for  Speech  Recognition, 
5th  International  Conference  on  Spoken  Language  Processing  (ICSLP  98), 

3:  1047-1050. 

Malfait,  M.  1996.  Using  Wavelets  to  Suppress  Noise  in  Biomedical  Images,  in  Aldroubi,  A.  and  M. 
Unser  (Eds.),  Wavelets  in  Medicine  and  Biology,  CRC  Press. 

Mallat ,  S.  1989.  A  Theory  for  Multiresolution  Signal  Decomposition:  The  Wavelet  Decomposition, 
IEEE  Transactions  on  Pattern  Analysis  and  Machine  Intelligence,  1 1 :  674-693. 

Mallat,  S.  1998.  A  Wavelet  Tour  of  Signal  Processing,  Academic  Press. 

Miller,  B.  L.  and  D.  E.  Goldberg  1995.  Genetic  Algorithms,  Tournament  Selection,  and  the  Effects 
of  Noise,  Complex  Systems,  9:193-212. 

Noel,  S.  and  H.  Szu,  1998.  Wavelets  and  Neural  Networks  for  Radar,  Progress  in  Unsupervised 
Learning  of  Artificial  Neural  Networks  and  Real-World  Applications,  Russian  Academy  of 
Nonlinear  Sciences,  ISBN  0-620-23629-9. 

Rajoub,  B.  A.  2002.  An  Efficient  Coding  Algorithm  for  the  Compression  of  ECG  Signals  Using  the 
Wavelet  Transform,  IEEE  Transactions  on  Biomedical  Engineering,  49(4):  355-362. 

Saha,  S.  2000.  Image  Compression  -  from  DCT  to  Wavelets:  A  Review,  ACM  Crossroads,  ACM. 

Saito,  N.  1994.  Simultaneous  Noise  Suppression  and  Signal  Compression  using  a  Library  of 
Orthonormal  Bases  and  the  Minimum  Description  Length  Criterion,  in  Foufoula-Georgiou,  E.  and 
P.  Kumar  (Eds.),  Wavelets  in  Geophysics,  Wavelet  Analysis  and  Its  Applications  Vol.  4,  Academic 
Press. 

Schilling,  D.,  P.  Cosman,  and  C.  Berry  1998.  Image  Recognition  in  Single-scale  and  Multiscale 
Decoders,  Proceedings  of  the  32nd  Asilomar  Conference  on  Signals,  Systems,  and  Computers, 

IEEE  Computer  Society  Press. 


42 


Schremmer,  C.,  T.  Haenselmann,  and  F.  Bdmers  2001.  A  Wavelet  Based  Audio  Denoiser,  IEEE 
International  Conference  on  Multimedia  and  Expo  (ICME)  2001,  145-148. 

Shannon,  C.  E.  and  W.  Weaver  1964.  The  Mathematical  Theory  of  Communication,  The  University 
of  Illinois  Press. 

Stein,  E.  M.  and  G.  Weiss  1971.  Introduction  to  Fourier  Analysis  in  Euclidean  Spaces,  Princeton 
Mathematical  Series  32,  Princeton  University  Press. 

Vaidyanathan,  P.  P.  1992.  Multirate  Systems  and  Filter  Banks,  Prentice-Hall. 

Yetterli,  M.  and  J.  Kovacevic  1995.  Wavelets  and  Subband  Coding,  Prentice-Hall. 

Villasenor,  J.,  B.  Belzer,  and  J.  Liao  1995.  Wavelet  filter  evaluation  for  image  compression,  IEEE 
Transactions  on  Image  Processing,  4(8):  1053-1060. 

Walker,  J.  S.  1999.  A  Primer  on  Wavelets  and  their  Scientific  Applications,  CRC  Press. 

Wang,  Y.,  Q.  Wu,  K.  R.  Castleman,  and  Z.  Xiong  2001.  Image  Enhancement  Using  Multiscale 
Oriented  Wavelets,  Proceedings,  IEEE  International  Conference  on  Image  Processing  (ICIP'01), 
Thessalonica,  Greece. 

Zhu,  Y.  and  S.  Schwartz  2002.  Discriminant  Analysis  and  Adaptive  Wavelet  Feature  Selection  for 
Statistical  Object  Detection,  Proceedings,  16th  International  Conference  on  Pattern  Recognition 
(ICPR'02),  4:  40086-40089,  IEEE. 


43 


